\chapter{Search Implementation}
In this chapter we will focus on how the search is implemented and used in the application. First we will expand the search requirements proposed in the theoretical part of this thesis. We will follow with the algorithms used to search for a stored informations.

\section{Search requirements}
The general requirements on search engines are to find document based on triples and terminologies that it contains. But we can provides much more richer search possibilities then only document search:

\begin{itemize}
	\item \textbf{Search for triples} - by writing only subject, predicate, object or any combination of these parts we would like to get which triple in the database contain these parts. We can call it as specifying a search query for document search. User does not need to have the knowledge of the stored triples and therefore do not need to try and guess the full triple for document search
	\item \textbf{Search for resource} - this will allow to search for nay resource having particular textual representation or lemma. Its a test on the database, if it contains document with that kind of knowledge. This will not result into the document search, but only in the terminology and triple parts search
	\item \textbf{Document search based on a triple} - Search based on full triple or any subpart
	\item \textbf{Document search based on a terminology} - Search based on the terminology, the results needs to be sorted by the TF-IDF to provide more relevant search results first
	\item \textbf{Document search based on a triples, specified by a terminology} - Since parts of the triples have mostly one word, we need to provide a user a way how to specify more his search query. Example triple: \emph{Smith works department}, department is a relatively general meaning. What we are trying to accomplish is that user will add \emph{sales department} in the terminology search field. We will then trigger search for documents having this triple and this terminology in its knowledge graph. 
\end{itemize}

As we have mentioned earlier, if the database contains a specific resource, it also contains its lemma and textual representations, therefore users do not have to write the URIs of the resources. The search will be based on the textual representation and lemmas.

\section{Implementation}
The search methods are available in the module \emph{dbVirtuoso} in the class \emph{DatabaseService} that implements \emph{IDatabaseService} interface, see figure\ref{fig:SearchDB}.

\begin{figure}[ht]
\centering
\includegraphics[width=0.7\textwidth]{./img/IDatabaseService.png}
\caption{Database Service Interface}
\label{fig:SearchDB}
\end{figure}

Method \emph{storeDocument} which is part of the interface is described in the previous chapter and it transforms the extracted informations into graph and stores it in the database.

\subsection{Search for triples}
The search is provided by method \emph{List<ITripleDbItem> searchTriples(String subject, String predicate, String object)} and it works in two steps:
\begin{enumerate}
	\item Determine which parts of the triple are specified
	\item For each part find the resource URIs that has textual representation or lemma same as input text
	\item If one part is specified then
		\begin{enumerate}
			\item For each resource URI
				\item Replace the ?subject, ?object, ?predicate string with the resource URI in the database query template \emph{SELECT DISTINCT * WHERE { GRAPH ?graph { ?subject ?predicate ?object}}} 
				\item Process the result and add triples into the resulting triples set
		\end{enumerate}
	\item else
		\begin{enumerate}
			\item For each combination of resource URI on two specified positions of the triple
				\item Replace the ?subject, ?object, ?predicate in the template
				\item Process the result and add triples into the resulting triples set
		\end{enumerate}
	\item return the set of triples
\end{enumerate}

The triples are then shown in the GUI. The triples returned are not just URIs, but they contain all the resource details (textual representations and lemma).  

\subsection{Search for resource}
User can test whether or not a resource exists in the database. By specifying any of the triple part or terminology, the application takes this text, creates query to search for a resource URI in the terminology or term or predicate graphs. If a resource has been found, then it extracts all details about this resource and returns them to the user. For terminology the method is \emph{List<ITerminologyDetail> getTerminologyDetail(String terminologyName)} and for part of the triple \emph{List<IResourceDetail> getResourceDetail(String resourceName)}.

\subsection{Document search based on a triple}
The search can be performed by filling all parts of the triple or by any subparts of the triple:

\begin{enumerate}
	\item Determine which parts of the triple are specified
	\item For each part find the resource URIs that has textual representation or lemma same as input text
	\item If no resource has been found then return null
	\item For each each part and each resource
		\begin{enumerate}
			\item Replace the ?subject, ?object, ?predicate string with the resource URI in the database query template \emph{SELECT DISTINCT ?graph WHERE { GRAPH ?graph { ?subject ?predicate ?object}}} 
			\item Extract the graph URI from the results and add it into the unique set of graphs
		\end{enumerate}
	\item For each matched graph extract all details about the graph, triples, terminologies, document file, plain text, statistics
	\item return the result
\end{enumerate}

We have decided to extract all information about the matched graph, so the user is able check the result if it suits him and download the original document that has been used for the information extraction.

\subsection{Document search based on a terminology}
On top of the document search based on a terminology, we can provide a way how to sort the result based on the terminology TF-IDF value. TF-IDF value represent how relevant is the word for a particular document, the higher the number is the more relevant is for the document. Therefore we are able to sort the document results based on their relevancy for a given term. The TF-IDF is not stored in the document for each terminology, because the IDF part might change for every new stored document in the database. Algorithm:
\begin{enumerate}
	\item Find resource with the same textual representation or lemma
	\item Create a triple, resource URI, predicate hasFrequency and object is unspecified. Predicate hasFrequency can appear only in document graphs, the resource URI on subject position also in the terminology graph
	\item Apply the query and search for the graphs.
	\item For each graph extract the details and calculate the TF-IDF for the terminologies
	\item Sort the graphs by the TF-IDF of the terminology resource specified in the query
	\item Return the graph set
\end{enumerate}

Result is then returned to application GUI.

\subsection{Document search based on a triple and a terminology}
As mentioned in the search requirements, we would like to provide a search based on the triple, but with a more specified constraint based on the terminology. For the given triple is searches for documents having this triple in its graph, then it searches for documents based on the given terminology. Both results are disjointed and only those documents being in both results are returned. The result is then sorted by the TF-IDF value of the terminology.